Repository navigation
Fit curves of named dose-response models in each cell type - #85
Merged
Merged
Conversation
A moderated test says that a gene responds to a dose, not what shape the response takes, while reports of transcriptomic dose response name a curve: BMDS's eight models, drc's log-logistic curve or DRomics' curves that rise and then fall. fit_dose_curves fits the sets "epa", "ll4" and "dromics" to a matrix of values under constant variance, and dose_response(models=..., variance=...) to every gene of each cell type, also by generalised least squares on the dose-group means of the cell type's moderated fit. Each fit starts from the lowest points of a fixed grid of its nonlinear parameters, with the linear ones solved exactly, and is refined within the model's bounds. Each gene gets every model's parameters, AICc, Akaike weight, F-test of lack of fit and largest scaled residual, its best model, where a bell turns and, from 6 dose groups, whether one dose group drives the turn; DoseResponseResult gains the table "curves", curve_fits and curve(model=...), and plot_dose_response draws model curves. The curves need no seed, do not depend on threads, and take about 3.7 ms per gene with the eight EPA models.
… limma and R validation/dose_curves.py fits five simulated series and genes built to turn, and runs pybmds 25.2, drc 4.0-0, DRomics 2.6.3, a 200-start scipy search, limma and R's anova on the same data in programs of their own: maxima, parameters, AIC penalties, test 4 from the F-test, lack-of-fit F-tests, the data of the weighted modes, turns and the dose group that drives each turn. dose_response_calibration.py gains rows of shape recovery and plotting.py the drawn model curves. --write-fixtures writes the fixtures that the tests read.
…d references The tests write every mean from the chapter's formulas, check gradients against central differences, likelihoods, AICc, weights and lack-of-fit tests by hand, the rules of fits not made, bounds and turns, invariants of order, units and threads, and the fixtures of pybmds, drc, DRomics, scipy and limma; dose_response's curves are checked against fit_dose_curves, limma and R, and the figure against the documented formula.
The chapter's section on curves of named models gives each set's formulas and bounds, the variance modes, how fits are found, how to read the curves table and turns, and how BMDExpress's outputs map; Choosing a method, the glossary, the bibliography, the API reference and the guide gain the curves, and the Validation page the recorded runs.
…ields #84 made the cube and the design keyword fields of DoseResponseResult, passed by pickling after the five positional ones; the curves join them as a keyword field, and the test of the fields' types follows. The guide was executed again with the curves' sections and the gate's in one notebook: every output of both is unchanged.
…tion page After the rebase onto #84, the calibration's section and the page's total held placeholders for runs that never finished, which fails the tests of the page's openings and totals, and the chapter's description had grown to 218 characters, beyond the 200 a link preview shows. The calibration's section now names the two runs whose rows it shows, on 2026-11-21: 139 checks without the rows of the curves and 111 without those of the gates, which printed the 101 rows they share alike, 149 checks in all. The total adds to 3,394 checks, none failed. The chapter's description lists its sections in their order, in 192 characters.
The plan puts the gate's call beside each gene's curves, and PR 9 left that to whichever of the two PRs merged second, but the curves table had no such column, so a reader had to join it with the table "gate" to keep the curves of the genes that pass. table(g, "curves") now holds passes_gate after the gene's ids in every cell type where the gate ran, matched to the table "gate" by feature id; cell types without the gate, because its test did not run there or because they support only description, keep the table as it was. The chapter's table of columns and its reading of turns name the column. A test checks the column against the table "gate" gene by gene on a series where both calls occur, and its absence in a cell type whose spline cannot run and in one that supports only description.
The search module's docstring still described a refinement that moves every parameter by Levenberg-Marquardt, from before the refinement became Kaufman's form of variable projection, and the docstring of the turn's dose divided the slope by b twice. The module docstring now says that the refinement moves the nonlinear parameters along their projected Jacobian columns, solves the linear ones again at every point tried, and tries each parameter alone before it stops; the turn's slope is written once, with dz/dx for both forms.
The script's docstring said that 36 genes go to the references under constant variance and that the weighted modes compare a spread of 6 genes with limma, while the script sends 28 genes (12, 8 and 8 of the three series), compares every gene's means, covariance and variances with limma, and gives scipy's searches 4 genes of each cell type. The docstring now says so, as the Validation page does; docstrings do not count in the run records' digests.
The helper's docstring still named a series that supports only description, which a test of its own now builds; the helper builds the weighted curves of one series.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
A moderated test says that a gene responds to a dose, not what shape the response takes or how sure that shape is. Reports of transcriptomic dose response name a curve: the genomic dose-response approach of the National Toxicology Program and EPA's Transcriptomic Assessment Products (ETAP) fit eight models in BMDS or BMDExpress, linear, polynomial, power, Hill and exponential 2 to 5, and choose one per gene (NTP 2018; US EPA 2024); pharmacologists fit drc's four-parameter log-logistic curve (Ritz et al. 2015); and DRomics adds Gauss-probit curves for responses that rise and then fall (Larras et al. 2018; Delignette-Muller et al. 2023). Until now CytePool gave a spline with a band and no named curve.
Each of those tools fits a gene's samples with their own variance and no weights, from starting values of its own, and none fits curves per cell type. Starting values and optimizers matter: on the series of this PR's validation, pybmds 25.2's maximum fell below that of the same model in 44 of 152 fits, drc 4.0-0's in 17 of 20 even with
relTol = 1e-12, and DRomics'nlsfits failed for 80 of its items and families and fell below for 13 of the 55 compared. Three replicates also strain the usual lack-of-fit test: BMDS's test 4 compares a likelihood ratio with a chi-square, which at nominal 0.10 rejects a correct straight line in 25% of experiments at 9 doses x 3 and in 38% at 9 doses x 2 (exact). A curve fitted without a spare dose group cannot test its own shape, and benchmark doses from such fits were non-protective in up to 80% of some four-group scenarios (Ringblom et al. 2014). A bell can rest on one outlying dose group, which EFSA found behind most apparent non-monotone responses (EFSA 2021).The solution
dose_response(models=...)fits a set of named models to every gene of each cell type, andcytepool.stats.dose_curves.fit_dose_curvesto a matrix of values outside a cube:"epa": linear, polynomial_2, power, hill and exponential_2 to exponential_5 in BMDS's forms, letters and bounds, written in dose units, with c of exponential 4 and 5 at most 1,000, CytePool's own bound where BMDS sets none."ll4": drc's LL.4 with b > 0 the slope, d the level at dose 0, c the level at high doses and e the EC50, DRomics' letters; drc's (b, c, d, e) with b < 0 are CytePool's (-b, d, c, e)."dromics": linear, exponential, ll4 and the Gauss-probit and log-Gauss-probit curves, each four- and five-parameter form a model of its own."constant", the default: each gene's values in the cell type's fit refitted against dose under one variance, as BMDS fits a gene's samples."weighted"and"moderated": the dose-group means of the cell type's moderated fit by generalised least squares with their covariance, which carries voom's weights and the design's other terms, and the residual or moderated variance (Pinheiro et al. 2014). A design with terms besides dose needs the weighted modes.models_not_fitted); exponential 2 to 5 are not fitted to a gene with a dose-group mean at or below 0, and no model to a gene whose values do not vary (not_fitted_reason).turn_dosegives where a Gauss-probit curve's slope changes sign strictly between 0 and the top dose, from its closed form; a curve that peaks at the top dose rises over every dose. For each gene whose best fit turns, the whole set is refitted without each dose group in turn, andturn_driven_by_dosegives the lowest dose without whose group the best fit no longer turns, 0 for the vehicle's. The check needs a model that can turn with a spare group among the K - 1 groups left, so from 6 dose groups; at 5,turn_driven_by_doseis NaN and the printouts say that the turns were not checked.passes_gatefollows each gene's ids in the curves table, matched to the table"gate"by feature id, so that curves cover every filtered gene andcurves.filter(curves["passes_gate"])keeps those of the genes that pass; cell types without the gate have no such column.Why it matters
Named curves are where CytePool's toxicological analysis starts. PR 11 computes benchmark doses and their bounds from these fits, PR 12 reads EC50s and Hill slopes from the log-logistic set, PR 13 averages models with the Akaike weights computed here, PR 14 applies NTP's and ETAP's rules to the
"epa"set, and PR 24 calibrates a test of the curves. The PR builds on PR 6's simulator, whose series the validation and the guide fit, and on PR 7'sdose_response, whose moderated fit of each cell type gives the weighted modes their means, covariance and variances. It sits on PR 8's contrast tests and PR 9's gate (#84): each cell type runs its tests and its gate, then fits its curves, with the vehicle's contrast added to the same fit; the curves table carries the gate's call, which PR 9 left to the later of the two PRs, and the chapter sends readers to the gate for which genes respond. Because each fit starts from a grid rather than from guesses, a gene's curve needs no seed and does not depend on the number of threads, and the validation can hold every reference's maximum as a lower bound.Validation
validation/dose_curves.py(new) simulates five dose series with PR 6's simulator: the vehicle and 8 half-log doses x 3 in two cell types, one of which loses its top dose; 4 decades x 3; 3 decades x 2; a cell type of single samples at 5 doses, which supports only description; and a design with sex. For turns, it adds genes that turn, or do not, by construction (a raised dose group, bells over three groups, rises at the top dose alone, flat genes) at 5, 6 and 9 dose groups under constant variance and at 9 and 5 in the weighted mode. Each reference runs as a program of its own on the same doses and values, with each model's mean and bounds written in the script from the documentation. A fit's maximum may not fall below a reference's by more than 1e-8 of the residual variance (or of Q / (K s^2) in the weighted modes), and parameters of fits that reach the same maximum off every bound, where the data identify them, are compared in standard errors scaled by the likelihood's gap.least_squares"epa"maxima not below pybmds's better direction (152 fits)anovaof nestedlmfits"ll4"maxima not below drc's (20); parameters (3)LL.4"dromics"lowest AICc of each family per sample (55); chosen model's parameters (27); the turn at DRomics' estimates (1)drcfitvoomDeliberate differences: pybmds completed no fit in 3 of its 304 runs (each fit then completed in the other direction), and its maximum lay below CytePool's in 44 of 152 fits (polynomial 2 17, Hill 11, exponential 5 9, exponential 3 4, power 3), by up to 9.0 log-likelihood units, and never beyond CytePool's bounds: pybmds's own exponential 5 fits went to c of 41, 69 and 183 though its table of priors lists [0, 20], so CytePool bounds c at 1,000, where its fits of those three genes run along the ridge to c near 1,000 with maxima above pybmds's. drc's maximum lay below in 17 of 20 fits. DRomics' fits failed for 80 of its items and families and lay below, beyond the rounding of its printed criteria, for 13 of the 55 compared; one Gauss-probit family lay 0.02 below in AICc where DRomics reports no parameters, and scipy's search within CytePool's bounds found no higher maximum there. 330 of CytePool's 27,712 fits reached 500 iterations before the stopping rule held, nearly all in the weighted and moderated modes, sliding along flat valleys, and keep the best point found; the scipy rows, which cover 4 genes of each cell type, set and mode, find every fit they cover at the maximum.
pybmds's PyPI wheel lacks NLopt's Luksan LBFGS and falls back to another optimizer when a fit asks for it. A probe that ran each pybmds fit in a process of its own found that 234 of the 304 runs attempted LBFGS; they completed as often as the others (the 3 incomplete runs had not attempted it), and their better direction fell below CytePool's maximum in 35 of 139 fits, against 6 of the 13 fits that never tried LBFGS. These fits give no sign that the fallback causes pybmds's lower maxima. This PR compares no BMDs; PR 11 should compare them only where pybmds reaches the maximum.
dose_response_calibration.pygains rows of shape recovery: on the power datasets of 9 dose groups with 3 replicates, the best fit's change from the vehicle to the top dose had the true sign, or turned for a bell, for 0.729 to 0.735 of about 2,600 responsive genes under"epa"in every variance mode, 0.736 under"ll4"and 0.766 under"dromics", whose best fits turned for 0.637 of the true bells but rested on one dose group in 0.960 of the flat genes' turns; PR 8's 101 rows came back identical.plotting.pygains two rows for the curves of a model, against drc's log-logistic formula evaluated in R: 5.9e-15 for the drawn curves and 4.0e-15 for their points, exact.Recorded runs before the rebase onto #84, each the second of two runs in one container (4 CPU, 8 GiB, AMD Zen 3):
dose_curves.py4,454 s and 1,030 MiB (its references take most of it: 200-start scipy searches of every fit, and 100-start refits of the whole"dromics"set without each dose group for the 47 turning genes),dose_response_calibration.py3,211 s and 455 MiB,plotting.py32.3 s and 883 MiB.dose_curves.py --benchmarkfit 10,000 genes in 37.4 s with"epa"and 35.7 s with"dromics", anddose_responsewith weighted curves took 30.2 s on a cell type of 10,000 genes against 0.58 s without curves; on the same kind of host before the reviews' changes, 41.6, 34.0 and 35.7 s. The benchmark's baseline is the ordinary least squares fit, while the performance targets use the weighted fit oflinear_models.py --benchmark, which a probe on an AMD Zen 3 host timed at 0.200 s: the curves cost about 190 such baselines with"epa"and 180 with"dromics", against the target of 350. The calibration script's 111 rows are those of its previous run but its two"dromics"rows, whose shares moved by 0.001 (turns now count strictly within the doses). The fixtures were byte-identical in the recorded run and the run before it. The helper also lists scripts whose checked code this PR changes without changing what they compute, as they importcytepool.plotting,cytepool.dose_responsesorcytepool._group_fits, whose expression filter moved into one function:activity.py,combined_p_values_and_replication.py,cube_collection.py,differential_expression.py,differential_expression_options.py,dose_axes.py,dose_contrasts.py,dose_design.py,dose_gate.py,dose_response_tests.py,dream.py,gene_set_files_and_ora.py,gene_set_tests.py,gsea.py,interferon_stimulated_pbmcs.py,kenward_roger_and_f_tests.py,meta_analysis.py,pathway_scores.py,pca_and_dot_plots.py,rotation_gene_set_tests.py,sumrank.pyandvariance_partitioning.py.The rebase onto #84 (PR 9's gate) merged only the two PRs' additions: a merge of the two heads differs from the rebased branch only in
dose_responses.pyand the calibration script, where each definition that both PRs changed holds both sides' changes and nothing else. Short runs at the merged code, in place of second recorded runs, show every script's rows unchanged. The CytePool side ofdose_curves.py(its series, every fit, the inputs of each reference and the turn tasks, 423,805 values) is bit-identical to that of the recorded code, and the script is unchanged, so its references see the same inputs. On a few datasets of every design cell of the calibration, eachdose_responseresult and permutation check of its rows was bit-identical at #84's code and at the merged code (153 calls, 1,237,226 values with the printed rows), and those of its curve rows at the recorded code (10 calls, 375,125 values; #84'sn_genes_passing_gatecolumn is the only addition).plotting.py,dose_gate.pyanddose_contrasts.py, run once at the merged code, printed every row of their recorded runs: 32, 26 and 80 checks, none failed. The Validation page gives the calibration's 149 checks from its two recorded runs, #84's 139 and this PR's 111, which printed their 101 shared rows alike, and a total of 3,394 checks. At the merged code,docs/changed_since_run.pylists the 39 scripts it lists on master and four more:dose_curves.pyanddose_response_calibration.py, whose records are of the runs above, anddose_gate.pyanddose_contrasts.py, which this PR's code reaches.The last four commits add
passes_gateto the curves table and correct docstrings. The table is built when asked for, and the scripts read onlyfeature_idand named parameter columns from it, while docstrings do not count in the digests, so the helper lists the same 43 scripts. The same probes, run again at the final head on the machine that ran them at 93bedc2, found the CytePool side ofdose_curves.py(423,805 values) and the calibration's curve rows (10 calls, 375,135 values) bit-identical.Modal runs, all ephemeral apps in
cytearc_profilingthrough the shared harness:dose_curves.pywhile writing itpr10-dromics-turnspr10-curves-baselinelimma_voomwith its observation weights) in one runpr10-docs-builddose_curves-write-fixtures(review)dose_curves.pyafter the reviews' changes, before the recorded runs, writing the fixturespr10-review-guide-avx512,-2(review)pr10-review-dose-curves-twice,pr10-review-calibration-plotting-twice(review)--write-fixturespr10-review-guide-avx512-3,-4andpr10-review-docs-build(review)pr10-rebased-docs-build(rebase)plotting,dose_gate,dose_contrasts(merged code)pr10-final-docs-buildpr10-verify-docs-buildpr10-verify-full-suiteTests
tests/test_dose_curves.pyand the curve tests oftests/test_dose_responses.pyandtests/test_plotting.pycheck:validation/dose_curves.py: maxima not below scipy's 200-start optimum in every set and mode, pybmds's and drc's maxima with their AIC penalties, test 4 from the F-test, scaled residuals and parameters, DRomics' criteria and the doses of its turns, and 47 verdicts of the dose group that drives a turn by the script's own coding of the rule, 14 of them at 5 dose groups; the means, covariance (M V M') and variances of the weighted modes and their lack-of-fit p-values against limma, rebuilt from the fixture's counts; the vehicle's mean at the average of the samples the fit uses."gate"on a series where both calls occur, and no such column in a cell type whose spline cannot run or that supports only description.Deliberate bugs, each applied to the source and caught by the test named:
test_constant_maxima_reach_scipys_best_of_200_startstest_constant_maxima_reach_scipys_best_of_200_startstest_weighted_curve_data_match_limma[ladder abundant voom-weighted]test_linear_and_polynomial_are_least_squarestest_linear_and_polynomial_are_least_squarestest_aicc_and_akaike_weights_by_handtest_aicc_and_akaike_weights_by_handtest_aicc_and_akaike_weights_by_handtest_lack_of_fit_by_handtest_lack_of_fit_by_handtest_locations_reach_beyond_the_dosestest_a_turn_below_the_lowest_dose_countstest_turn_doses_are_extrema_of_the_documented_mean,test_a_turn_below_the_lowest_dose_countstest_a_turn_can_rest_on_the_vehicletest_turns_and_the_group_they_rest_ontest_declinestest_linear_and_polynomial_are_least_squarestest_noise_free_curves_are_recovered[epa-hill]test_noise_free_curves_are_recovered[dromics-gauss_probit]test_gauss_probit_bounds_holdtest_the_vehicles_mean_is_at_the_average_of_the_other_termstest_descriptive_values_are_those_a_fit_computestest_weighted_curves_use_the_fits_means_and_variancestest_model_curvestest_turns_at_five_dose_groups_are_not_checkedtest_a_peak_at_the_top_dose_is_not_a_turntest_bounds_are_reportedtest_descriptive_cell_types_get_curves_without_p_valuesdose_responsetest_a_column_of_sample_groups_is_checked_before_curvestest_model_curvestest_plot_dose_response_draws_model_curvestest_arguments_are_refused_with_a_fixtest_plug_in_likelihood_aicc_and_residuals_by_handtest_constant_scaled_residuals_by_hand,test_epa_fits_reach_pybmds_maxima_with_its_statisticstest_the_lower_of_two_driving_doses_is_named,test_turn_drivers_follow_the_rule_with_refits_of_the_samplestest_the_data_without_a_group_are_those_of_its_samplestest_the_table_puts_the_turn_driver_on_the_best_rowtest_weighted_curves_with_a_sample_left_outtest_the_curves_table_carries_the_gates_callpasses_gateafter the other columns instead of after the gene's idstest_the_curves_table_carries_the_gates_call,test_constant_curves_fit_the_samples_values,test_the_curves_table_carries_symbolsCoverage:
dose_curves.pyis covered in full, and so are this PR's lines ofdose_responses.py,plotting.pyand_group_fits.py; withNUMBA_DISABLE_JIT=1, every line and branch of the compiled kernels in_dose_curve_models.pyand_dose_curve_search.pyruns. The branches that no input could reach (a grid without a finite point, a step with no free parameter) were removed, and values whose sums of squares would overflow, beyond 1e100 in size, are refused.At the final head, the full suite passed with 6,242 tests and 18 skipped (zstd needs Python 3.14), on this machine and on Modal, where 18 more skip because the container has no git checkout and runs as root; ruff, ruff format and mypy were clean.
Decisions
variance="constant"(default),"weighted","moderated""reference";"residual"for"weighted""weighted"says how the means are weighed, and PRs 11 and 17 use itpolynomial_2restricted to coefficients of one sign, the better of both signs; both directions of the exponential models"epa"from BMDS in dose units, with levels unbounded"ll4"in DRomics' letters, b > 0 and d at dose 0; drc's (b, c, d, e) with b < 0 is (-b, d, c, e)passes_gatein the curves table where the gate ran, after the gene's ids and matched by feature id"gate"; a column of False where the gate did not rundose_responses.py; the parts offit_dose_curvesthatdose_responseruns are listed as sharedtests/test_private_names.py), and lists such shared parts, assumrankandvariance_fractionsdoTrimmed, with where each went: ETAP's Hill rule and AIC selection under a rule set to PR 14; curves of
dream_voomfits to PR 21 (constant variance would ignore the random intercepts, and the weighted modes need degrees of freedom that dream's fits do not keep); bands of model curves to PRs 11 and 12, with profile-likelihood bounds; DRomics' selection filters, not reproduced, since PR 9's gate decides which genes respond; BMDS's non-constant and log-normal variance, out; exponents below 1 in power and Hill, not offered, as BMDExpress and ETAP restrict them; lists of models to PRs 13 and 21.